Deploying machine learning (ML) solutions to cloud infrastructures can be problematic. In particular, deploying ML solutions to an application platform-as-a-service (aPaaS) is a formidable challenge. For example, there is an impedance mismatch between underlying assumptions in aPaaS architectures, and the requirements of a multi-tenant ML solution. More plainly stated, the aPaaS tends to be geared for transaction-processing systems, which typically are more I/O-bound, while ML systems are more compute-bound.
In some examples, an aPaaS packages server software into self-contained, stateless virtual machines (VMs) with minimal computational resources assigned to each (e.g., single-core, small virtual CPU, and limited memory). The VMs should not store any data, and instead delegate persistence to database services adjacent to the aPaaS environment. Any files stored inside the VM's virtual disk gets removed when it is restarted. However, this multiple, stateless, small VMs paradigm is a source of difficulty when deploying ML services. Notably, the need for long running periods of compute-bound processing, and having large client-dependent state does not readily fit the aPaaS paradigm.